Holographic microscopy of holographically trapped three-dimensional structures

ABSTRACT

A method and system for performing three-dimensional holographic microscopy of an optically trapped structure. The method and system use an inverted optical microscope, a laser source which generates a trapping laser beam wherein the laser beam is focused by an objective lens into a plurality of optical traps. The method and system also use a collimated laser at an imaging wavelength to illuminate the structure created by the optical traps. Imaging light scattered by the optically tapped structure forms holograms that are imaged by a video camera and analyzed by optical formalisms to determine light field to reconstruct 3-D images for analysis and evaluation.

CROSS REFERENCE TO RELATED PATENT APPLICATIONS

This application claims the benefit under 35 U.S.C. 119(e) of U.S.Application 60/897,784 filed Jan. 26, 2007, incorporated by referenceherein in its entirety.

This invention is directed to a holographic optical trapping systemusing optical traps generated by computer-established holograms toorganize materials and apply microscope optics to inspect and analyzethe materials in three dimensions (3-D). More particularly, aholographic video microscope system uses real-time resolved volumetricimages of 3-D microstructures to carry out analysis and inspection ofmaterial assemblies.

The U.S. Government has certain rights in this invention pursuant togrants from the National Science Foundation through Grant NumberDBI-0629584 and Grant Number DMR-0606415.

BACKGROUND OF THE INVENTION

Holographic optical trapping uses computer-generated holograms to trapand organize micrometer-scale objects into arbitrary three-dimensionalconfigurations. No complementary method has been available in the priorart for examining optically trapped structures except for conventionaltwo-dimensional microscopy. Three-dimensional imaging would be usefulfor a variety of uses, such as verifying the structure ofholographically organized systems before fixing them in place. It alsowould be useful for interactively manipulating and inspectingthree-dimensionally structured objects such as biological specimens.Integrating three-dimensional imaging with holographic trapping mightseem straightforward because both techniques can make use of the sameobjective lens to collect and project laser light, respectively.However, conventional three-dimensional imaging methods, such asconfocal microscopy, involve mechanically translating the focal planethrough the sample. Holographic traps, however, are positioned relativeto the focal plane, and would move as well. The trapping pattern wouldhave to be translated to compensate for the microscope's mechanicalmotion, which would add substantial complexity, would greatly reduceimaging speed, and would likely disrupt the sample undergoingexamination and analysis.

SUMMARY OF THE INVENTION

Digital holographic microscopy solves all of the prior art technicalproblems, providing real-time three-dimensional (3-D) imaging datawithout requiring any mechanical motion, including no need to translatethe focal plane through the sample under analysis. A particularlycompatible variant of in-line holographic microscopy replaces theconventional illuminator in a bright-field microscope with a collimatedlaser. Light scattered out of the laser beam by the object interfereswith the remainder of the incident illumination to produce a heterodynescattering pattern that is magnified by the objective lens and recordedwith a video camera. This scattering pattern is a hologram of thetrapped structure. Provided that this interference pattern is notobscured by multiple light scattering, it contains comprehensiveinformation on the scatterers' three-dimensional configuration. Eachtwo-dimensional snapshot in the resulting video stream encodestime-resolved volumetric information that can be analyzed directly, ordecoded numerically into three-dimensional representations. This systemand method enables ready commercial use of digital holographicmicroscopy in a holographic optical manipulation system, and uses thecombined capabilities to directly assess both techniques' accuracy andestablish any limitations.

Various detailed aspects of the invention are described hereinafter, andthese and other improvements and features of the invention are describedin detail hereinafter, including the drawings described in the followingsection.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a system constructed in accordance with theinvention;

FIG. 2A illustrates a conventional bright-field image of five colloidalspheres trapped in an xy plane (the scale bar is 5 micrometers); FIG. 2Billustrates the pattern of FIG. 2A rotated about a y axis by 45°; FIG.2C illustrates a bright-field image of the rotated pattern of FIG. 2B asseen in the xy plane; FIG. 2D illustrates a coherent image of the samestructure as seen in the xy plane; and FIG. 2E illustrates a holographicreconstruction of an xz slice through the tilted pattern (circles denotethe intended particle coordinates);

FIG. 3A illustrates a hologram recorded in an xy plane of a singlesphere trapped at x=17 micrometers above a focal plane; FIG. 3Billustrates the real part of the scattered field reconstructed from FIG.3A; FIG. 3C shows a hologram recorded with the sphere at x=0; FIG. 3Dshows an axial section of the scattered field obtained by translatingthe subject colloidal sphere past the focal plane in Δ=0.122 μmmicrometer steps; FIG. 3E shows an equivalent reconstruction usingconventional illumination; and FIG. 3F illustrates axial intensityprofiles from FIG. 3B and 3D, demonstrating accuracy of the axialreconstruction; and

FIG. 4A shows resolution limits for occluded objects in the xy plane andFIG. 4B for the zy plane.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 1 shows a schematic representation of an integrated system 10constructed in accordance with the invention. The system 10 is based onan inverted optical microscope (such as, Zeiss Axiovert S100-TV)outfitted with a 100× NA 1.4 oil immersion objective lens 20. This lens20 is used both to project holographic optical traps, and also tocollect in-line holographic images of trapped objects. Holographic trapsare preferably powered by a frequency-doubled diode-pumped solid statelaser 25 (such as, a Coherent Verdi) operating at a wavelength of 532 nmto generate input laser beam 30. A liquid crystal spatial lightmodulator 35 (such as a Hamamatsu PAL-SLM X7550) imprints the beam'swavefronts with phase-only holograms encoding the desired trappingpattern. The modified trapping beam 40 then is relayed to the inputpupil of the objective lens 20 and is focused into optical traps.

The trapping beam 40 is preferably relayed to the objective lens 20 witha dichroic mirror 50 tuned to the trapping laser's wavelength. Otherwavelengths pass through the dichroic mirror 50 and form images on a CCDcamera 60 (such as, NEC TI-324AII). In a most preferred embodiment astandard combination of incandescent illuminator and condenser lens 62has been replaced with a helium-neon laser providing 5 mW collimatedbeam of coherent light at a wavelength of λ=632 nm in air. The system 10further includes a computer 65 for manipulation of sensed image data andanalyzing the image data by executing calculations of all equationsprovided herein by conventional software known in the art. The computer65 can also include any conventional executable memory, such as a ROM,RAM, or other well known memory capable of storing a program, data orother instructions which can be executed to fulfill the analyzationfunctions described herein.

FIG. 2A demonstrates holographic imaging of colloidal spheres 70holographically trapped in a three-dimensional pattern. These 1.53 μmdiameter silica spheres 70 (Bangs Labs Lot No. L011031B) are dispersedin a 50 μm thick layer of water confined within a slit pore formed bysealing the edges of a #1.5 cover slip to the surface of a clean glassmicroscope slide. Each of the spheres 70 is trapped in a separatepoint-like optical tweezer, and the individual optical traps arepositioned independently in three dimensions. FIG. 2A shows aconventional bright-field image of the sphere or the particles 70arranged in a focal plane. Projecting a sequence of holograms with thetrapping positions slightly displaced enables us to rotate the entirepattern in three dimensions, as shown in FIG. 2B. As the particles 70move away from the focal plane, their images blur, as can be seen inFIG. 2C. It is difficult to determine from this image whether the mostdistant particles 70 are present at all.

FIG. 2D shows the same field of view, but with laser illumination. Eachof the particles 70 appears in this image as the coherent superpositionof the laser light it scatters with the undiffracted portion of theinput laser beam 30. Other features in the image result fromreflections, refraction and scattering by surfaces in the optical trainof the system 10. These can be minimized by subtracting off a referenceimage obtained with no particles or trapped structure in the field ofview.

Enough information is encoded in two-dimensional real-valued images suchas FIGS. 2A-2E to at least approximately reconstruct thethree-dimensional complex-valued light field. The image in FIG. 2E is anexample showing a numerically reconstructed vertical cross-sectionthrough the array of particles 70. This demonstrates the feasibility ofcombining holographic microscopy with holographic optical trapping. Thereconstruction is consistent with the anticipated 45° inclination of thearray, and with the calibrated 5.9 μm separation between the particles70. Intended particle coordinates are shown as circles superimposed onthe image. This quantitative comparison demonstrates the utility ofholographic microscopy for verifying holographic assemblies. Becauseholographic images, such as FIG. 2D, can be obtained at the full framerate of the video camera 60, holographic microscopy offers the benefitof real-time data acquisition over confocal and deconvolutionmicroscopies.

In a most preferred embodiment, very accurate results can be obtainedfrom use of the Rayleigh-Sommerfield formalism because holograms, suchas in FIG. 2D form at ranges comparable to the light wavelength. Thefield u(r, z)) scattered by an object at height z above the microscope'sfocal plane propagates to the focal plane, where it interferes with thereference field, a(r), comprised of the undiffracted portion of thelaser illumination. The Rayleigh-Sommerfeld propagator describing theobject field's propagation along optical axis 80 is:

$\begin{matrix}{{{h_{z}(r)} = {{- \frac{1}{2\pi}}\frac{\partial}{\partial z}\frac{^{\; {kR}}}{R}}},} & (1)\end{matrix}$

where R²=r²+Z² and k=2πn/λ is the light's wavenumber in a medium ofrefractive index n. The field in the focal plane is the convolutionu(r,0) {circle around (X)}h_(z)(r). The observed interference pattern,therefore, is

I(r)=|a(r)|²+2

{a*(u{circle around (X)}h _(z))}+|u{circle around (X)}h _(z)|²   (2)

The first term in Eq. (2) can be approximated by measuring the intensitywhen no objects are in the field of view. FIG. 2D was obtained bysubtracting such a reference image from the measured interferencepattern. If we further assume that the scattered field is much dimmerthan the reference field, the second term in Eq. (2) dominates thethird. In that case,

$\begin{matrix}\begin{matrix}{{b(r)} = {\frac{{I_{0}(r)} - {{a(r)}}^{2}}{{a(r)}} \approx {2\; \frac{\left\{ {a*\left( {u \otimes h_{z}} \right)} \right\}}{{a(r)}}}}} \\{\approx {2\left\{ {u \otimes h_{z}} \right\}}}\end{matrix} & \begin{matrix}(3) \\\; \\(4)\end{matrix}\end{matrix}$

provides a reasonable basis for reconstructing u(r). Ghosting can beminimized by translating trapped structures away from the focal plane.

Analyzing Eq. (3) can be simplified by assuming a(r)=1 for the referencefield. In our application, however, the illuminating laser trapping beam40 passes through an inhomogeneous sample before reaching the focalplane. Any resulting amplitude variations can be eliminated bynormalizing I(r) with |a(r)|. Structure in the illumination's phasecannot be compensated in this way, and must be assumed to vary moregradually than any features of interest.

Reconstructing the three-dimensional intensity field is most easilyperformed using the Fourier convolution theorem, according to which

$\begin{matrix}\begin{matrix}{{B(q)} \equiv {\int_{- \infty}^{\infty}{{b(r)}{\exp \left( {{- }\; {q \cdot r}} \right)}{^{2}r}}}} \\{{\approx {{{U(q)}{H_{z}(q)}} + {{U^{*}(q)}{H_{z}^{*}(q)}}}},}\end{matrix} & \begin{matrix}(5) \\\; \\(6)\end{matrix}\end{matrix}$

where U(q) is the Fourier transform of u(r, 0) and

$\begin{matrix}{{H_{z}(q)} = {\exp\left( {\; {{kz}\left\lbrack {1 - \frac{\lambda \; q^{2}}{2\pi \; n}} \right\rbrack}^{\frac{1}{2}}} \right)}} & (7)\end{matrix}$

is the Fourier transform of the Rayleigh-Sommerfeld propagator.

The estimate for the Fourier transform of the object field at height z′above the focal plane is obtained by applying the appropriateRayleigh-Sommerfeld propagator to translate the effective focal plane:

B(q)H_(−z′)(q)≈U(q)H_(z−z′)(q)+U*(q)H_(−z−z′)(q)   (8)

The first term in Eq. (8) is the reconstructed field, which comes intobest focus when z′=z. The second is an artifact that is increasinglyblurred as z′ increases. Unfortunately, this term creates a mirror imagearound the plane z=0 with the result that objects below the focal planecannot be distinguished from objects above. This ghosting is apparent inFIG. 2E.

Our final estimate for the complex light field at height z above thefocal plane is

$\begin{matrix}\begin{matrix}{{\upsilon \left( {r,z} \right)} \equiv {{{\upsilon \left( {r,z} \right)}}{\exp \left( {{\varphi}\left( {r,z} \right)} \right)}}} \\{= {\frac{1}{4\pi^{2}}{\int_{- \infty}^{\infty}{{B(q)}{H_{- z}(q)}{\exp \left( {\; {q \cdot r}} \right)}{^{2}q}}}}}\end{matrix} & \begin{matrix}(9) \\\; \\(10)\end{matrix}\end{matrix}$

Equation (9) can reconstruct a volumetric representation of theinstantaneous light field in the sample under inspection from a singleholographic snapshot, I(r). The image in FIG. 2E is a cross-sectionthrough the reconstructed intensity distribution, v(r,z)².

Each sphere in FIG. 2E appears as a bright axial streak centered on arelatively dark dimple at the object's three-dimensional position.Circles superimposed on FIG. 2E denote the intended three-dimensionalpositions of the spheres 70, which were used to compute the trap-forminghologram that arranged the spheres 70. The very good agreement betweenthe optical traps' design and features in the resulting reconstructedfield attests to the accuracy of both the projection and imagingmethods.

Contrary to previous reports in the prior art, images such as those inFIGS. 3A to 3F suggest that the axial resolution of our holographicreconstruction approaches the diffraction-limited in-plane resolution.FIG. 3A shows a hologram obtained for one of the spheres 70 held by anoptical tweezer at height z=17 μm above the focal plane. FIG. 3B is anaxial section through the real part of field reconstructed from

{v(r,z)}=|(r,z)| cos (Φ(r, z)). This representation has the benefit ofmost closely resembling the scattering field observed in conventionalthree-dimensional bright-field microscopy. The sphere, in this case, iscentered at the cusp between bright and dark regions. This crossover inthe scattered field's sign creates a dark dimple in the intensity.

The effective axial resolution can be assessed by scanning the spherepast the focal plane and stacking the resulting images to create avolumetric data set. FIG. 3C is a hologram of the same sphere from FIG.3A at z=0. Compiling a sequence of such images in axial steps of Δ=0.122μm yields the axial section in FIG. 3D.

Structure in the spheres' images along the axial direction can beanalyzed to track the spheres 70 in z, as well as in x and y. For themicrometer-scale particles or the spheres 70 studied here, for example,the centroid is located in the null plane between the downstreamintensity maximum and the upstream intensity minimum along thescattering pattern's axis. Holographic microscopy of colloidal particlestherefore can be used to extract three-dimensional trajectories moreaccurately than is possible with conventional two-dimensional imagingand far more rapidly than with scanned three-dimensional imagingtechniques. In particular, in-plane tracking can make use ofconventional techniques, and tracking in depth requires additionalcomputation but no additional calibration.

Analyzing images becomes far more challenging when objects occlude eachother along the optical axis, as FIGS. 4A and 4B demonstrate. Here, thesame pattern of the spheres 70 from FIGS. 2A-2E has been rotated by 90°,so that four of the spheres 70 are aligned along the optical axis 80.FIG. 4A is a detail from the resulting hologram and FIG. 4B is theholographic reconstruction in the vertical plane of the structure. Thecentral observation from FIG. 4B is that all four of the spheres 70 areresolved, even though they directly occlude each other. A fifth sphere70, not directly occluded by the others was included as a reference, andis visible to the right of the others in FIGS. 4A and 4B.

The uppermost spheres 70 in FIG. 4B appear substantially dimmer thanthose trapped closer to the focal plane; and FIGS. 4A and 4B compensatefor this by presenting the amplitude |v(r,z)|, rather than theintensity, of the light field. The reference sphere 70, however, is nobrighter than its occluded neighbor, and no dimmer than any of thespheres 70 in FIGS. 2A-2E. Rather, the lower spheres 70 act as lenses,gathering light scattered from above and focusing it onto the opticalaxis 80. As a result, these spheres 70 appear substantially brighterthan normal, and their images are distorted. Equation (9) does not takesuch multiple light scattering into account when reconstructing thelight field.

The resulting uncertainty in interpreting such results can be mitigatedby acquiring images from multiple focal planes, or by illuminating thesample under investigation from multiple angles, rather than directlyin-line. Results also would be improved by more accurate recordings.Each pixel in our holographic images contains roughly six bits of usableinformation, and no effort was made to linearize the camera's response.The camera 60 was set to 1/2000 s shutter speed, which nonethelessallows for some particle motion during each exposure. A wider dynamicrange, calibrated intensity response and faster shutter all wouldprovide sharper, more accurate holograms, and thus clearerthree-dimensional reconstructions.

With these caveats, the image in FIG. 4B highlights the potentialimportance of holographic imaging for three-dimensional holographicmanipulation. The most distant particle 70 appears displaced along theoptical axis relative to the reference particle even though both werelocalized in optical tweezers projected to the same height.Three-dimensional visualizations confirm the structure of the projectedtrapping field. The apparent axial displacement was not evident forinclinations less than roughly 80°. It therefore reflects either athree-dimensional imaging artifact or, more likely, a real displacementof the particles 70 from their designed configuration. This isreasonable because light from the traps projected closer to the focalplane exerts forces on particles trapped deeper into the sample. Thiseffect is exacerbated by particles trapped closer to the focal plane,which deflect light onto more distant particles, altering theireffective potential energy wells. This effect has been exploited forin-line optical binding of particles trapped along thread-like Besselbeams. Holographic imaging provides a means for measuring suchdistortions, and thus a basis for correcting them. This can becritically important for processes such as the holographic assembly ofphotonic heterostructures which rely on accurate placement of microscopesuch particles or other objects.

The foregoing description of embodiments of the present invention havebeen presented for purposes of illustration and description. It is notintended to be exhaustive or to limit the present invention to theprecise form disclosed, and modifications and variations are possible inlight of the above teachings or may be acquired from practice of thepresent invention. The embodiments were chosen and described in order toexplain the principles of the present invention and its practicalapplication to enable one skilled in the art to utilize the presentinvention in various embodiments, and with various modifications, as aresuited to the particular use contemplated.

1. A method of performing 3-D holographic microscopy of an optically trapped structure, comprising the steps of: providing an optical microscope; generating a laser beam having an associated imaging wavelength input to the inverted optical microscope; generating a plurality of optical traps with wave fronts of a trapping laser beam having a phase only hologram encoding a desired optical trapping pattern and the trapping laser beam having an associated trapping wavelength; and providing an objective lens for the inverted optical microscope, the objective lens both projecting the plurality of optical traps and collecting in-line holographic images of the trapped structure; and providing images of the trapped structure to a CCD camera for three-dimensional holographic microscopy of the trapped structure.
 2. The method as defined in claim 1 further including the step of interposing a dichroic mirror between the objective lens and the CCD camera, the dichroic mirror tuned to the trapping laser beam's trapping wavelength.
 3. The method as defined in claim 1 wherein the images include ghosting which can be minimized by translating the trapped structure from the focal plane.
 4. The method as defined in claim 1 wherein the imaging laser beam is provided by a helium-neon laser having a collimated beam output.
 5. The method as defined in claim 1 wherein the inverted optical microscope includes a focal plan and the method involves no mechanical motion for performing three-dimensional holographic microscopy of the trapped structure.
 6. The method as defined in claim 1 further including the step of subtracting off a reference image from the image formed at the CCD camera, thereby removing a varying background illumination when none of the trapped structure is present.
 7. The method as defined in claim 1 wherein the images of the trapped structure include two-dimensional real-valued images for reconstruction of a three-dimensional complex-valued light field.
 8. The method as defined in claim 1 including the additional step of analyzing the images using a Rayleigh-Sommerfeld formalism.
 9. The method as defined in claim 8 wherein the Rayleigh-Sommerfeld formalism includes analyzing propagation of the trapped structure along an optical axis of the inverted optical microscope.
 10. The method as defined in claim 9 wherein the trapped structure gives rise to a scattered field u(r,z) at a distance z upstream of a focal plane of the optical microscope and the scattered field u(r,z) is reconstructed by $\begin{matrix} {{b(r)} = {\frac{{I_{0}(r)} - {{a(r)}}^{2}}{{a(r)}} \approx {2\frac{\left\{ {a*\left( {u \otimes h_{z}} \right)} \right\}}{{a(r)}}}}} \\ {\approx {2\left\{ {u \otimes h_{z}} \right\}}} \end{matrix}$
 11. The method as described in claim 1 further including the step of reconstructing a 3-D light field v(r,

) of the images by solving, $\begin{matrix} {{\upsilon \left( {r,z} \right)} \equiv {{{\upsilon \left( {r,z} \right)}}{\exp \left( {{\varphi}\left( {r,z} \right)} \right)}}} \\ {= {\frac{1}{4\pi^{2}}{\int_{- \infty}^{\infty}{{B(q)}{H_{- z}(q)}{\exp \left( {\; {q \cdot r}} \right)}{^{2}q}}}}} \end{matrix}$
 12. The method as defined in claim 11 wherein the light field v(r,

) is reconstructed from a single holographic snapshot of the image.
 13. The method as defined in claim 12 further including the step of tracking movement of the trapped structure.
 14. The method as defined in claim 12 wherein the trapped structure comprises a plurality of objects which are occluded and the three-dimensional images are reconstructed and the objects all resolved in the reconstructed light field.
 15. The method as defined in claim 1 further including the step of acquiring image data from multiple focal planes of the optical microscope, thereby enhancing accuracy of the images of the trapped structure.
 16. A system for performing three-dimensional holographic microscopy of an optically trapped structure, comprising: an inverted optical microscope; a laser source for producing a trapping laser beam; a spatial light modulator for providing a phase-only hologram for imprintation on the trapping laser beam; a laser source for producing an imaging laser beam an objective lens associated with the inverted optical microscope; a CCD camera for detecting laser light arising from imaging of the trapped structure and the CCD camera outputting image data; a computer for analyzing the image data using computer software executed by the computer.
 17. The system as defined in claim 16 wherein the laser source comprises a laser which produces a collimated beam of coherent light.
 18. The system as defined in claim 16 further including a dichroic mirror disposed between the objective lens and the CCD camera.
 19. The system as defined in claim 16 wherein the computer software comprises mathematical formalisms including Rayleigh-Sommerfeld determinations of light field v(r,

) for the laser beam received by the CCD camera.
 20. The system as defined in claim 16 wherein the Rayleigh-Sommerfeld determinations comprise embedded computer software executed by the computer to calculated, $\begin{matrix} {{\upsilon \left( {r,z} \right)} \equiv {{{\upsilon \left( {r,z} \right)}}{\exp \left( {{\varphi}\left( {r,z} \right)} \right)}}} \\ {= {\frac{1}{4\pi^{2}}{\int_{- \infty}^{\infty}{{B(q)}{H_{- z}(q)}{\exp \left( {\; {q \cdot r}} \right)}{^{2}q}}}}} \end{matrix}$ 